Don't live in fear and hide from these people. Confront the scumbags who are trying to confront you, and hit them with this:
"Thank you for your concerns brothers, but I have private & personal issues which I can't discuss with anyone at the moment, but I certainly appreciate your motives and concern. If things change, I know that I can call you."
They'll get the message.